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SUMMARY 


This is a position paper introducing the concept of 
remote presence in telemanipulation. Remote presence or 
telepresence is a property of an intimate man/machine 
interface in which the human operator is provided with a 
simulated sense of physical presence at the remote task 
site. It is suggested as an alternative to supervisory 
control for optimizing performance. Evidence is cited to 
support the contention that enhancement of a sense of 
presence will improve performance. A conceptual design of 
a prototype teleoperator system incorporating remote 
presence is described. The design is presented in 
functional terms of sensor, display, and control 
subsystems. The concept of an intermediate environment, in 
which the human operator is made to feel present, is 
explicated. The intermediate environment differs from the 
task environment due to the quantity and type of 
information presented to an operator and due to scaling 
factors protecting the operator from the hazards of the 
task environment. Several research issues pertaining to 
the development of a telepresent manipulator are 
delineated. The potential benefits of remote presence 
systems, both for manipulation and for the study of human 
cognition and perception are discussed. 


INTRODUCTION 


Remote manipulators, or teleoperators, are devices 
designed to allow the performance of manipulative tasks in 
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environments that are either too hostile or too remote to 
permit the physical presence of a human being. Situations 
in which use of remote manipulators may be appropriate 
include handling of highly radioactive materials and 
construction or exploration in space or undersea 
environments. 

Recently efforts have been made to make the human 
operator (HO) of the device remote not only from the 
environment of the manipulation task, but also from direct 
control of the actual manipulations. The KO is 
increasingly being placed in a supervisory or "higher 
level" control loop (1). The trend to define the 
operator's function as that of supervisory controller has 
been based on a combination of physical constraints and 
assumptions concerning human performance. The HO in direct 
control of a remote manipulator, equipped with industry 
standard visual and end point displayed force reflecting 
feedback and vise-like grip , does not perform remote 
manipulation tasks as accurately or as quickly as those 
tasks performed by direct hands-on manipulation (2,3). The 
assumption is that a semi- or completely automated sensor 
or algorithm driven manipulator could surpass the HO 
performance. The physical or psychomotor constraints 
requiring supervisory control are ennunciated by Ferrell 
and Sheridan (4): 

a) the HO is so physically removed from the device 
as to cause inherent time delays in the control 
or feedback loop due to transmission lags. Such 
transmission lags could also be introduced in 
cases where a hostile environment necessitated 
bandwidth limitations on feed forward or feedback 
transmission , 

b) the operator is overburdened with other control 
or d-cision tasks. 

c) the operator is prevented from exercising 
direct control, either due to environmental 
constraints of space limitations or as a result of 
physical handicap , as in the case of 
quadriplegia. 

Under many conditions physical or psychomotor 
constraints are not the limiting factor. If the 
allocation of supervisory function is based on previous 
performance deficits or on HO overburdening resultant from 
inappropriately displayed information, we contend that the 
operatot should be placed in direct control of the remote 
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manipulator. Rather than removing the operator from tho 
control loop efforts should be made to more tightly couple 
the HO and the manipulator. 

The performance values of the HO should be exploited. 
The HO is adaptive and able to respond to anomaly. The 
operator's neuromuscular response provides for rapid, 
varied and fine control of manipulation. Direct control 
serves to reduce computational costs both in time and 
range of function. Finally, tasks performed in an 
environment whose characteristics are unstructured or 
unknown (and given the state of the art of computational 
intelligence) require that the HO be relied on for 
control . 


METHOD 


We have noted that in present systems the HO displays 
poor performance when uirect control of a remote 
manipulator. Thi** method of control is currently 
characterized by limited or inappropriately coded 
information to the HO and by a technical subsytem of 
limited dexterity. We contend that this deficit is a 
function of the method by which control is effected rather 
than an inherent limitation of the HO in the performance of 
the task. 

Examination of the literature (5) has led us to the 
conclusion that tightening the loop between the op'erator 
and effector will provide adequate to superior performance 
in manipulation. The conclusion is supported by three 
’ ^nes of evidence. 

1) As previously reviewed, the supervisory control 
paradigm is appropriate when a tight link between 
operator and effector cannot be maintained. Such 
an approach is necessitated by the fact that 
continuous control is not possible if the 
HO/effector time lag increases beyond the HD's 
reaction time (6). 

2) Physiological research in neuromuscular control 
indicates that tight sensory motor integrat'on is 
necessary for functional motor control (7, 8, 9, 
10 ). 

3) Inference from previous research indicates that 
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as the link between operator and manipulator is 
tightened performace improves. Master slave 
mechanical linkage with force reflecting feedback 
improved performance (11). Tight mechanical link 
with the effector improves performance (12). 

We propose that the operator/effector link can be 


tightened through the development of high 
integrated displays, true master 
incorporating multiarticulated end 
appropriate transformation algorithms, 
the conceptual design parameters 
development for each of these subsystems 


fidelity sensors, 
slave control 
effectors, and 
Consideration of 
and recommended 
fol lows. 


CONCEPTUAL DESIGN 


Figure 1 represents a potential conceptual design for 
a telepresent manipulation system. The arm is designed to 
be a position feedback system with force proportional 
control. A review of the state of the art of the 
subsytems indicated has recently been made (5) and will not 
be detailed here. The subsystems will be reviewed with 
attention to the requirements of an effective man/machine 
interface and to the research issues which need to be 
addressed for system realization. 


Sensor Subsystem. 


The sensor subsytem of the proposed advanced 
manipulator system serves the dual function of sensing 
operator input and commands through the master arm and 
sensing the condition of the remote arm, including 
environmental influences on that arm. 

Position sensors referencing the condition of the 
master arm are well within the state of c * art. We have 
in Figure 2 indicated that a simple rotary potentiometer is 
sufficient to provide the necessary control information. 

A more chullenging aspect of the sensor subsystem are 
those sensors which are to provide information about the 
remote arm and the environment in which it is operating. 
Several types of sensors in several modalities are 
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suggested. Tactile information, including touch, slip, and 
pressure is required. Several technologies have developed 
which may provide the necessary capabilities. They have 
been reviewed in detail by Harmon (13). Strain gouges (14) 
have been considered. Pressure sensitive materials (15) 
have been tested. Hill and Bliss (16) have tested a polymer 
switch arrangement in manipulator control. Bejczy (17, 
18) has developed many of the advanced tactile sensor 
systems which would be useful In manipulator control, 
including touch sensors, pressure ser.sors and directional 
slip sensors. This sensory information is to be used to 
drive a distributed and veridical display to the operator. 
It is suggested that the sensor density follow roughly the 
human sensitivity to these inputs, with highest density 
sensor distribution in the end effector and a significant 
reduction in density for the more proximal portions of the 
mani pul ator . 

Information regarding the forces impinging on the 
manipulator and end effector must be transmitted to the 
operator for proper control. These forces must be sensed on 
the manipulator and localized in terms of their directon 
and magnitude. Judiciously placed strain guages, for 
example, are well suited to this task. 


Sensor Research Issues 


Sensors used to characterize a hostile environment 
must be shielded from that environment wihtout loss of 
sensitivity to changes in environmental conditions. Such 
selective ruggedization of sensors must be explored. 
Recent experience indicates that it is possible that as 
analogs of biological systems are developed, they will be 
as prone to failure, in the same situation, as their 
living counterparts. 

Another issue for investigation is determination 
of the type of sensory information most useful to the 
operator. This issue is highly interactive with display 
capabilites. Our design suggests incorporation of as many 
sensory modalities as possible. 


Display Subsystem 


Of significantly greater difficulty than acquiring 
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tactile and propriocerti ve information is effectively 
displaying it to the operator. Propriocepti ve information 
should be displayed to the KO as a distributed function 
with torques felt about the appropriate axes of rotation. 
If the load is at the endpoint of the manipulator, as in 
hand grip, then loading should be displayed to the hand of 
the operator. If, on the other hand, the loading is 
distributed over the entire surface of the manipulator, as 
in large load lifting or encounter with distributed 
resistance, the loading m.ust be displayed over the 
re1ev?nt surfaces of the operator's arm. The current 
procedure of localizing all feedback at the wrist imposes a 
cognitive load in force translation and a fatiguing 
physical load at the operator's wrist. An exoskeltal 
position feedback system is suggested to provide 
distributed proprioceptive information. The distributed 
surface pressure display (Figure 4) responds to surface 
sensor activity at the remote location. The display will 
provide distributed touch sensing ability to the operator. 
In addition the sleeve provides for temperature display via 
hydraulic chambers responsive to temperature sensors at the 
remote site. 

Tactile information has classsically been displayed 
via mechanical stimulation through vibrotactile 
transducers. Vibrotactile stimulation has oeen induced by 
airjets, piezoelectric vibratory elements, electromagnetic 
and electromechanical stimulation (16, 19, 20). 
El ectrocutaneous stimulation has been explored (21). 
Electrocutaneous stimulation can be used to provide 
sensations of pressure, pain and heat. Flexibility of 
stimulation parameters, portability, and increasing 
miniaturization potential recommend electrotacti le 
information display. 


Display Research Issues 


Both electrotacti le and vibrotactile stimueltion share 
certain research issues which constellate around questions 
of: 

- How accurately does the sensation provided 
mimic natural stimulation? 

- How do the percieved attributes of the 
sensation vary as a function of body site 
stimulated? 

> What is the optimal display density and 
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distribution? 


In addition to the more common display issues there 
are several psychophysical interactions which could be 
exploited in the effective presentation of electrotacti le 
stimulation. Structural interactions provide stability 
and accuracy in the inherently noisy neural systems. For 
example, lateral inhibition in the retina is a process of 
local inhibition of receptors as a function of stimulation 
intensity. This inhibition provides a sharpening of the 
illumination difference among the receptors. The result i; 
a perceptual demarcation of stimulus intensity change that 
has no physical corollary in the stimulus. The sharpening 
of tactile stimuli by the simulation of the biological 
transduction process of structural interaction has been 
initiated (22). Temporal interactions in tactile display 
also provide an ability to exploit physiological 
processing. Sensory saltation (23) and the "phi 
phenomenon" (24) both, through appropriate stimulation 
sequencing, provide stable tactile stimulation which is not 
associated with a particular stimulator site. These 
processes suggest methods of increasing display density 
without a cost in operator encumbrance. 


Control Subsystem 


The control system is intended , as illustrated in 
Figures 2 and 3a,b,c, to be a fully articulated master 
slave control. The master slave control concept is likely 
to maintain the tight link between operator and manipulator 
which is necessary to generate a sense of presence. The 
computational requirements in the master slave controller 
should be significantly lower than other types of 
control 1 ers . 


Control Research Issues 


The integration of several interactive feedback and 
feedforward loops which is required in our design raises 
some issues asociated with control stability. It is 
imperative that the operator control link be maintained as 
tightly as possible so as to minimize control lags and to 
maximize system stability. 


t. / 
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One factor which must be investigated in the control 
system is that of translation between an operators hand or 
arm motion and that of the manipulator. A disparity 
between these motions is inherent in the system design. 
For example, the hand controller not only translates the 
operator's motion commands to the manipulator, but also 
acts as a tactile and proprioceptive display to the 
operator. The physical requirements of the display limit 
the operator's range of motion. This limitation in hand 
closure, for instance, must be accounted for in the command 
structure to the manipulator. 


Translation Subsystem. 


The example cited above regarding control translation 
from somewhat limited operator movement to full 
manipulator motion is just one of a class of issues which 
we have designated as requiring a translation subsystem. 
The environment for whicn the manipulator is designed 
cannot be displayed directly to the operator. Similarly 
the control functions of the operator must be translated 
to appropriate patterns for manipulator kinematics. The 
translation is conceived to take place in an environment 
which is intermediate between the hostile or remote 
environment and the actual operator control area. 

The translations must take into account: 

- actenuating or filtering hostile environment 
i nfl uences , 

- scaling sensory input to an appropriate range 
for the operator, 

- making some modality ^■ransformations , e.g., if 
the manipulator is expo.^d to damaging 
influences, the information mighi be 
reasonably transmitted to the the operator 

as a moderately uncomfortable heat sensation 
which the operator could choose to eliminate 
if there were need to continue to operate 
in that environment. 

- scaling the operator input to a range 
appropriate to the manipulator kinematics. 

- compensating for system induced distortions in 
control or feedback, e.g., overcoming the 
display/control distortion in the hand 
controller, or compensating for system 
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inertial properties in the exoskeleton 
operator/ i n ter face . 

The intermediate environment concept provides a 
mapping technique in which translation rules can be 
imposed in the feedforward and feedback loops of the 
manipulator system. The imposition of these translations 
creates an intermediate environment for man/machine 
interaction. 


CONCLUSION 


We propose the concept of an advanced manipulator 
system in which the operator is intimately linked to the 
system in a state of "remoted presence" as an alternative 
to current trends in teleoperator research. Examination of 
the required elements of a telepresent system has generated 
research issues which must be addressed before the 
prototype of such a manipulative device can be 
constructed: 

- The necessary density of sensor arrays must 
be investigated. 

- Given use in hostile environments, sheilding 
of sensor arrays without unduly limiting 
sensitivity or manipulative ability 

must be developed. 

- The capability of currently available tactile 
stimulation technology to provide simulation 
of touch, and the necessity of doing so, must 
be evaluated. 

- Applicability of such physiological phenomena 
as sensory saltation and inhibition to enhance 
tactile display capability must be assessed. 

-The algorithmic requirements of translating 
control and sensor information between the 
task and intermediate environments must be 
determined. 

- The functional environment, in which the 
operator is to be made to feel present, 
is referred to as the intermediate 
environment. The information content of 
that intermediate environment required for 
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optimal operator performance must be defined. 

- The mechanical subsystem must be made capable 
of the fine manipulations that high rjssolution 
sensor and display subsystems will support. 

- The complexity of the control system that 
integrates the sensor, display , control 
subsystems, and translation algorithm must 
be examined to determine overall system 
stability. 

The development of a telepresent manipulator system 
will contribute to performance in remote manipulation 
tasks approaching that of a truly present human operator. 
By immersing the operator so completely in this man-machine 
system, a remote system may for the first time possess the 
adaptability of the human being in unexplored environments 
and unstructured situations. If properly implemented, the 
master-slave configuration uf the system may substantially 
reduce the computational requirements of the control loop 
over those of a supervisory control system. Since a 
remote presence is resultant of the combination of 
displays of several sensory modalities (visual , auditory, 
tactile, and kinesthetic) and effector capability, it may 
provide an avenue for the study of human cognition and 
perception. In the most advanced state, the sensor/display 
systems of a telepresent manipulator could allow selective 
control over the stimuli presented to an operator. 
Directing research toward the development of such a system 
may result in both enhanced capability in remote 
manipulation tasks, and a powerful tool for the study of 
human psychomotor control. 
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FIGURE 1 TELEPRESEN"^ MAS'^ER/SLAVE MANIPULATOR 




FIGURE 2 MASTER ARM 


A) KASUIR ARM TORSIC.^AL JOINT LINKAGE 

CONTAINING ROIARY POTENTIOMETER 

B) SERVO MOTOR PROVIDING POSITION 
FEEDBACK 

C) WRIST FLEXION-EXTENSION- CONTROLLER 
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I-IGURE 3a r.Ul lIARUCUUlfiD IIASIKR HAND (TOP VIEW) 

A) FINGER THIMBLE AND ELECTROTACTI LE DISPLAY 

B) FINGER DISTROL RING 

C) THUMB DISTROL OUTRIGGER TUBE 
0) HINGED LOWER THUMB DISTROL 

E) TORQUE MOTOR MOUNTING PLATE (MOTORS NOT SHOWN FOR CLARITY^ 

F) KNUCKLE 3RACE 

G) WRIST FLEXION-EXTENSION DISTROL 
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F) KNUCKLE BRACE WITH EYELETS AND CABLE GUIDES 




FIGURE 4 MASTER ARM HtAT/PRtSSURE DISPLAY 

A) iraKGRATKO HKAT/PRl SSURE DjSPLAY POCKET 

B) hydraulic lUBING 

• HIGH TEMPERATURE FLUID SUPPLY 

• LOW TEMPERATURE FLUID SUPPLY 

• EXHAUST TUBING 

C) TUBES SUPPLYING PALM DISPLAY 

D) PALM DISPLAY PAD 
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